Search for: All records

Large Language Models (LLMs) are pre-trained on large-scale corpora and excel in numerous general natural language processing (NLP) tasks, such as question answering (QA). Despite their advanced language capabilities, when it comes to domain-specific and knowledge-intensive tasks, LLMs suffer from hallucinations, knowledge cut-offs, and lack of knowledge attributions. Additionally, fine tuning LLMs' intrinsic knowledge to highly specific domains is an expensive and time consuming process. The retrieval-augmented generation (RAG) process has recently emerged as a method capable of optimization of LLM responses, by referencing them to a predetermined ontology. It was shown that using a Knowledge Graph (KG) ontology for RAG improves the QA accuracy, by taking into account relevant sub-graphs that preserve the information in a structured manner. In this paper, we introduce SMART-SLIC, a highly domain-specific LLM framework, that integrates RAG with KG and a vector store (VS) that store factual domain specific information. Importantly, to avoid hallucinations in the KG, we build these highly domain-specific KGs and VSs without the use of LLMs, but via NLP, data mining, and nonnegative tensor factorization with automatic model selection. Pairing our RAG with a domain-specific: (i) KG (containing structured information), and (ii) VS (containing unstructured information) enables the development of domain-specific chat-bots that attribute the source of information, mitigate hallucinations, lessen the need for fine-tuning, and excel in highly domain-specific question answering tasks. We pair SMART-SLIC with chain-of-thought prompting agents. The framework is designed to be generalizable to adapt to any specific or specialized domain. In this paper, we demonstrate the question answering capabilities of our framework on a corpus of scientific publications on malware analysis and anomaly detection. 

For the third consecutive year, Scholarship for Service (SFS) scholars at the University of Maryland, Baltimore County (UMBC) analyzed the security of targeted portions of the UMBC computer systems. During these hands-on studies, with complete access to sourcecode, students identified vulnerabilities, devised and implemented exploits, and recommended mitigations. We report on our continuing experiences with these project-based learning studies, focusing on the new problems addressed in January 2018 and 2019 and on the lessons we learned. In 2018, students analyzed the WebAdmin custom software that UMBC students, faculty, and staff use to manage credentials and accounts. Students found a beautifully instructive example of a “confused-deputy attack,” wherein an IT staff member—–through carrying out their proper procedures for resetting a user password—–unwittingly executes malware on their own machine by viewing the answers to security questions. In 2019, students analyzed the Virthost system UMBC uses to host student webpages. Organizer Alan Sherman created a powerful learning experience by secretly recruiting one of the participants to serve as a “mole,” passively collecting passwords from the other participants throughout the week. Our students found the collaborative experiences inspirational; students and educators appreciated the authentic case studies; and IT administrators gained access to future employees and received free recommendations for improving the security of their systems.